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ON MULTI-RATE SEQUENTIAL DATA TRANSMISSION 



CHEUK TING LI 

Abstract. In this report, we investigate the data transmission model in which 
a sequence of data is broadcasted to a number of receivers. The receivers, 
which have different channel capacities, wish to decode the data sequentially 
at different rates. Our results are applicable to a wide range of scenarios. 
For instance, it can be employed in the broadcast streaming of a video clip 
through the internet, so that receivers with different bandwidths can play the 
video at different speed. Receivers with greater bandwidths can provide a 
smooth playback, while receivers with smaller bandwidths can play the video 
at a slower speed, or with short pauses or rebuffering. 



1. Introduction 

Consider the scenario in which a long video chp has to be transmitted to a 
number of receivers having different packet loss ratios. One approach is to divide the 
video data into blocks of K packets, encode each block into L> K encoding packets, 
and then transmit the blocks to the receiver sequentially. Using random linear 
projections or any capacity-achieving erasure code, the receiver can decode the block 
if about K out of L packets are received. This method, which we call a blockwise 
code, can only cater for the need of the receiver with packet loss probability less 
than 1 - K/L. 

To suit the need of different receivers, we can perform time multiplexing on two 
blockwise codes at different rates. Cosider Blockwise code 1 and Blockwise code 
2, which use random linear projections to encode each block of K packets into 
Li and L2 packets respectively {Li < L2)- Denote the i-th packet generated using 
Blockwise code k by Pk,i- We transmit the packets of the two codes in an interleaved 
manner (in the sequence i'l,!, -P2,ij -f'i,2: -f2,2i ^'1,3, •■■)• Receiver 1, which uses only 
the packets generated using Blockwise code 1, can decode a block using K out of 
the Li packets encoded from the block, and therefore can tolerate a packet loss 
probability 1 — K/Li. As Blockwise code 1 transmits a block of K packet per Li 
channel uses, taking interleaving into account, Receiver 1 can decode at a rate of 
K/{2Li) packets per channel use. Receiver 2 uses packets generated by both codes. 
It can decode a block using K out of the Li + L2 packets encoded from the block, 
and allows a higher packet loss probability 1 — K/{Li + L2). However, to use the 
packets generated by both codes. Receiver 2 has to wait for the slower Blockwise 
code 2, which transmits a block of K packet per L2 channel uses. Receiver 2 can 
decode at a rate if/ (21/2)- 

In the scenario, the receivers with different channel conditions wish to decode the 
same sequence of data. Each receiver will decode the data sequentially at a roughly 
constant rate which depends on the channel condition. We call these settings as 
multi-rate sequential data transmission. It can be viewed as multilevel diversity 
coding [T] with an additional sequential decoding constraint. In the following sec- 
tions, we will discuss various cases of the problem. The case of one transmitter is 
described in Section [3j The main contribution lies in Section [6] concerning multi- 
ple transmitters, in which multi-rate sequential data transmission has apparently 
dissimilar behavior compared to its non-sequential counterpart. 
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Figure 1.1. Encoding process of blockwise and multiplexed codes 

2. Formulation of Multi-rate Sequential Data Transmission 

We consider the transmission of a sequence of data through an erasure channel. 
To simplify the setting, we assume that one bit is sent at a time through the erasure 
channel. Assume the sender is going to transmit a sequence of bits Mi, M2, .... 

We assume Mj Bern(l/2). The sender will encode it into a sequence of bits 
Xi,X2,.-. and transmit them through an erasure channel. The symbols arrive 

at the receiver as Yi,Y2,..., where some of them may be erased (denoted by e, 
Yi E {0, 1. e}). Based on these symbols, the receiver tries to decode Mi, M2, .... 

Definition 1 (MRS code). A multi-rate sequential code (MRS code) is specified 
by a pair of encoding and decoding functions. The encoding function is a function 
mapping the message {Mi} to the encoding symbols {Xj}. There is a random 
variable Q supported in Q which is known by both the sender and the receiver, 
and independent of the message and the channel erasure, to allow random coding 
scheme. The encoding function is a function 

Enc : Q X {0,1}'*' x N ^ {0,1} 

{Q,{M^},J) ^ X,. 

Note that this definition allows the encoder to look at all blocks. 

The decoding function maps the received symbols {1^ }, where some of them may 

be erased, to the recovered blocks |-Mj|. Let y = UneNi^' l'^}" be the space of 
received symbols. The decoding function is a function 

Dec : Q X X N ^ {0, 1} 

(Q,yi",«) ^ m. 

We use the notation Y^ — {Ya, Ya+i, Yb). For simplicity, we write Xj {Q, {Mi})) = 
Enc (Q, {Mi} ,j), and Mi (Q, y^") = Dec {Q, Y{\ i). 

The MRS code does not admit a fixed rate like other block codes. Instead its 
rate depends on the channel capacity. 

Definition 2 (admissible pair). A rate-capacity pair (r. c) is called c- admissible by 
a code if there exist Nq such that when Xj — >■ Fj is an erasure channel with capacity 
c (i.e. erasure channel with erasure probability 1 — c), 

P |m„ ^ (g, Y^) I < e for any A/" > TVq and m < N{r ~ e). 

In other words, any receiver with channel capacity c can decode the first N{r — e) 
bits ''■"'^ with bit error probability less than e when the first N symbols Y^ 
are received, for sufficiently large N. 
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Figure 2.1. Rate-capacity functions of blockwise and multiplexed 
blockwise code 

It is clear that if (c, r) is e-admissible, then all pairs in {(c', r')\c' > c,r' < r} are 
e-admissible. Therefore we can use a function to characterize the rate of a code. 
We call r : [0, 1] — >■ [0, oo) a rate-capacity function if it is monotonically increasing, 
right continuous, and there exists an r; > such that r(c) = for c < rj. 

Definition 3 (rate of MRS code). A rate-capacity function r(c) is called e-admissible 
by a code if all of the pairs (c, r(c)) are e-admissible by the code. 

The rate-capacity functions of the blockwise code and the multiplexed blockwise 
code described in the introduction can be given by Figure |2.1[ 

Definition 4 (achievable rate-capacity functions). A rate-capacity function r(c) is 
achievable if for any e > 0, there exist a code where r(c) is e— admissible by that 
code. 



In this section, we will present the design of superposition multi-rate sequen- 
tial codes, and prove their optimality by giving the set of achievable rate-capacity 
functions explicitly. 

Definition 5 (superposition MRS code). A superposition MRS code is character- 
ized by the block size K and the parameter g : [0,oo) — > [0,cxd) which is bounded, 
monotonically decreasing and left continuous with g{a)da = 1. The message 
{Mi} is divided into blocks of K bits, Bi = Af^*/^-^^^^-^. In the encoding process 
of the code, we first sample a sequence of non-negative random variables Ai, A2, ... 
i.i.d. according to the cumulative distribution function 



Note that FA{a) is increasing as g{a) is decreasing. The sequence is known by 
both the sender and the receiver (we may let Q = {Ai,A2, ...)). At time instance i, 
the sender generate an encoding symbol Xi from the block B^i.^i/K] using random 
linear projections and transmit it to the receiver. 

Note that we employ a random coding scheme. Random linear projections allow 
us to decode a block of K bits using K -\- o{K) encoding bits with an arbitrarily 
small error probability. The superposition MRS code is essentially performing time 
multiplex on blockwise codes at different rates. 



3. Superposition Coding for Single Transmitter 



(3.1) 
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The parameter g{a) roughly corresponds to the proportion of the encoding sym- 
bols dedicated to satisfying the need of the receivers which wish to decode at rate 
a. When we send encoding symbols encoded at rate 7 (i.e. Ai = 7), a receiver 
which decodes at rate a < 7 can use a portion of a/7 of the encoding symbols. The 
parameter g{a) describes the proportion of the symbols which can be used, divided 
by the proportion needed (which is a), given by 

1 

9ia) - - / -dFAh), 

a Ja 1 



which can be verified using (3.1 1 



1 /"OO /"OO -1 

' -dFAil) = / -dFAil) 
7 Ja 1 

• 1 

- • 7^^5(7) 

7 

dg{l) = 9(.a)- 

The blockwise code and the multiplexed blockwise code described in the intro- 
duction are examples of superposition MRS codes. For the blockwise code, the 
parameter is taken to be 



3(a) 



L/K when a < K/L 
when a > K/L. 

For the multiplexed blockwise code, the parameter is taken to be 

r ki^ when a < ^f^ 

5(a) = <^ ^ when < a < ^ 

[0 when a > 2x7- 

The choice of the parameter is closely related to the rate-capacity function we 
would like to achieve. The following theorem describes the relationship between 
the two functions. 

Theorem 6. For a fixed e, the rate- capacity function r(c) is e-admissible by the 
superposition MRS code with block size K and parameter g{a) for all sufficiently 
large K , if there exists > satisfying 

c ■ g {r{c)) + for all c> with r{c) > 0. 

Proof. Fix e > 0. Let r(c) be a rate-capacity function, and let g{a) be a bounded 
and monotonically decreasing function (let g{a) < C for all a > 0). Assume 
the condition c • g {r{c)) > 1 + ^ is satisfied for some ^ > 0. We take K > 
max (4/^, 32 • (^ -I- l)/(e^)). We now consider the superposition MRS code with 
block size K and parameter g{a). 

Fix any channel capacity c with r(c) > 0. Let ai = ?'(c), and ao = r(c) — e. At 
time instance i, the sender generate an encoding symbol from the block ^^[i.Ai/if] 
(note that Bi = Af(\^i)i<-+i)- Let F4(a) = g{x)dx — ag{a). The probabihty 
that Bj is chosen is 



K{j - 1) Ki 

1 



i J \ i 



Let n > ^, and j < -I- 1. We will study whether the block Bj can be 
decoded using Y" with error probability less than e when the channel capacity is c. 
If so, then Af}"""^ can be decoded using Yj" with bit error probability less than e 
whenever n> a-nd thus the rate-capacity pair (ai,c) is e— admissible. Let 5* be 
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the random variable representing the number of times Bj is chosen in the encoding 
of Xi, and Xi is not erased, for i = 1, ...,n. Its expected value is given by 

Kj\ ^ fK{j-l) 



Note that is monotonically increasing and not greater than 1 (and therefore 
Fa (^) is monotonically decreasing with respect to x) , we have 



> n{ I FA{ — ]dx\-l, 



Therefore 

\Q] ^ r-r, f f f p. f ^ ^ 

nx J \ nx 



m > cn{ l' (FA(^]-FA(^^^^]]dx]-, 



where 



dx = FA{a) - / xdFA 

= FA{a)+a / -dFA{x) 
= Fa{q) -\- ot I — ■ —xdg{x) 

J a ^ 

POO 

= FA{a) - a ldg{x) 

J a 



= FA{a)+ag{a) 
g{x)dx. 



Hence, as g{x) is monotonically decreasing, 

/ i-Kj/n i-K(j-l)/n \ 

E[5] > cn • / g{x)dx - / g{x)dx 

\Jo Jo y 

i-Kj/n 

= cn- I g{x)dx — 1 

> cK-g(^ao+^-l 

> cK ■ g (ao + e) — 1 
= cK-g{ai)-l 

> K{l + ^)-l. 
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On the other hand, 



£-(f) < »(r-(s)-)' 



and 



Let 



dx 



> n{ I Fa{ ^^LJI ]dx]-l. 
nx 



Pi = c[FA 



i 

The variance of S can be given by 



Var[5] = J2iP^-P') 

i=l 
n 



< 



^ nx I \ nx 

1 



< n.( / \Fa'^\-fA'^^1^\\Ax 



nx J \ nx 

/ i-Kj/n rK{i-l)/n \ 

n-(y g2{x)dx- J g2{x)dxj+l 



n- I g2{x)dx+l 

JKd-D/n 



lK(j-l)/n 

< KC + 1, 

as<VK- VcTl. 

As a result, for K large enough, S is close to E[S] with high probability. Cheby- 
shev's inequality gives 

p|5<i^(l + 0-l-^^^^}<e/2, 

where 

> K {1 + ^/2) 

by the assumption K > max (4/^, 32 • (i^ + l)/(e^)). When random linear projec- 
tions arc used, for a fixed probability of error e/2, the number of symbols needed 
to decode a block of K bits is K + o{K), which is smaller than K {1 + ^/2) for 
sufficiently large K. Therefore the rate function r(c) is e-admissable by the super- 
position MRS code with parameter g{a) for K large enough. □ 
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The following theorem gives the achievable region for the rate-capacity function, 
which coincides with the region attained by superposition MRS code. 

Theorem 7. A rate-capacity function r(c) is achievable if and only if 

/ -dr{c) < 1. 
Jo c 

Proof of achievability. Assume r(c) is a rate-capacity function satisfying ^dr(c) < 
1. We can find 77 > such that r(c) = for c < rj. Let 3(0;) = l/r~^(a), where 
r~^{a) = inf{c|r(c) > a}. Then g{a) is monotonically decreasing and bounded 
above by l/rj. It is implied by ^dr{c) < 1 that g{a)da < 1. As r~^(r(c)) < c, 
we also have c • g {r{c)) > 1 for all c > with r(c) > 0. 

Fix e > 0. Note that g{a + e/2)da < 1. We can define a function g2{ce) by 

32(a) = (1 + •.g(« + e/2), 

where ^ > such that g2{a)da ~ 1. We know 32 is bounded above by {l + £)/rj. 
Define a new rate-capacity function r2(c) — max(r(c) — e/2,0). We have, for any c 
with r2(c) > 0, 

c-32(r2(c)) =c.(l + 05(Kc)) > 1 + e 

Consider the superposition MRS code with parameter f/2 {0) ■ By Theorem [6j 
the rate-capacity function r2(c) is e/2-admissible by the code for sufficiently large 
block size, which implies that r(c) is e-admissible by the code. □ 



Proof of converse. The proof employs a similar idea as in [2 . As r(c) is a rate- 
capacity function, we can find r/ > such that r(c) = for c < rj. For any e > 0, 
consider a code where r(c) is e-admissible. The message {Mi} are encoded into 
binary symbols {^i}, and sent through an erasure channel with capacity c (we call 
it Channel c) to give {^ci} for all c > 0. Assume we have the following for any c, 

max P |m„, ^ M,n (Q, | < e for any n>No. 

Let N > Nq. Let Ed be the indicator of the events of erasure in Chan- 
nel c {Ec.i — 1 means that Y^i = e)- Note that whether the message can be 
decoded at the receiver with Channel c depends only on the marginal distribu- 
tion of {Ec^i,Xi^ Mi,Q}^^^, and is conditional independent of Ec2,i for C2 c 
given {Ec^i, Xi^, Mi,Q}_^^j^. Therefore, we may modify the joint distribution of 
{£'c,i}(,g(o 1] iGN without affecting the result as long as the distributions of {Ed, Xi, Mi, 
are preserved. From now on, we assume the channels are cascaded, i.e. whenever 
Co < ci < 1, the Markov chain Xi — > Yci.i — > ico,i holds, and 

„ j£'ci,i w.p. co/ci 

w.p. 1 - Co/Ci. 

Let 

(3.2) r2(c) = max(r(c) - e,0). 
Define 

(3.3) /(c)= -^.i7(r,^i|Q,M|^'-^(=^£;,^i). 
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The common random variable Q will be omitted for simplicity. Consider Channel 
Co and Channel ci where 77/2 < cq < ci < 1. Let fco = \_Nr2{cQ)\, and ki = 
[Nr2{ci)\. Note that 



(3.4) = /^(F,^,l,M,^^Vll^^^<l)-^«^ll^^^<l)' 



where, due to the assumption that Mj are i.i.d. uniform in {0, 1}, 



^^«Vi|A^i'", = -f^Ko+i) = h-ko> iV(r2(ci) - r2(co)) - 1. 



As can be decoded using Yj^^ with bit error probability less than e, by 
Fano's inequality, 



< H{Yl,\M^o^El,) 
fci 



+ J2 H{M,\Yl„M^°,El,) 

i=feo + l 

< H{Yl,\M^«,El,) + (fci - fco) • H{e) 

< H{Yl,\M^o,El,) + (iV(r2(ci) - r^ico)) + 1) • H{e) 

< H{Yl,\Ml\El,) + {N{r,{c,) - r,{co))) ■ H{e) + 2. 



Let El ^ ■ Bern(l — Cq/ci) with 



1 if R' = 1. 



We can obtain 
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N 



(i) 



(ii) 



(iii) 



(iv) 



(v) 
> 



i=l 
N 



i=l 
N 



i=l 
JV 



i=l 
N 



4=1 

N 

E 

i=l 
N 

E 

i=l 

AT 



- E ( ? • H(Y^oAMi'',Y:-i,E^o., El = 0) 



i=l 
Co ^ 
Cl 



ci 



Cl 



where (i) isduetoff(F;-i|y;-i,i?i^^i) = 0, (ii) is due to (i^^-j, X Fco,»| (Mf°,r;-i), 

(iii) is obtained by conditioning on and by Fco,i = c when K- = 1, (iv) is 
due to El AL (r,, ,„M^,y;-i,i;,,,,), and (v) is due to H{El-l\Y^J) = and 

eI^^,aly,,J{m^'>,y:-1). 



Hence by (3.4) 



HiYl,\M^\El,) 

< ^ • H{Y,l,\M^°,El,) + {N{r,{c,) ~ r,{c„))) ■ H{e) + 2 
-(iV(r2(ci)-r2(co))-l), 



After replacing the terms by /(c) using (3.3 1, 



- (.2(ci) - r,{co)) < ^ ^^^^ 



/(co)-/(ci) + ^ ^ /(Co) - /(ci) + ^ 



1-Hie) 
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Due to the monotonicity of r2{c), 



-dr2(c) 



1 1 

Co Ci 



< — (r2(ci) - r2(co)) 

Co 



(r-2(ci) -r2(co)) 

(r2(ci) - r2(co)) 



1 1 

Co Ci 



Cl 



(r-2(ci) - r2(co)) 



< 



/(Co) - /(ci) + ^ 
l-i/(e) 



Let m 



dbij c — 5 • * • ) ^771 1 ^-nd 

1 1 



ivj , and Co = Cq < c'l < ... < c^ = Ci such that r2(c) is continuous 



2/1 1 
< — I ) for i = 0, m — 1. 



c- c'i+i m Vco Cl 



We can always find such c'^ as a monotonic function has at most countable 
discontinuities. Note that for i = 0, ...,m — 1, 



112 
Ci <+i m \co Cl 

as ?7/2 < Co < Cl. Then we have 



< 



mr] 



I 

J a 



Cl ^ 

-dr2{c) 



= E 



< 



< 



< 



i=0 

m— 1 

E 

i=0 



••<^^: + i 1 



dr2{c) 



./(c',)-/(c',+i) + ^ /I 1 



1 - /f(e) 



(r2(c^+i)-r2(cO) 



/(Co) - /(Cl) + % , 4 

— r3ij(e^ — + -(-2(^i)--^(^o)) 

6m . 



/(co)-/(ci) + ^ 



mi] 



-r(l). 



Note that 



/(c) = ^.^^(n^ilQ,^/^^-^^)-'^^,) 



< 1. 



Thus we have, for any rj/2 < Cq < Ci < 1, 

_|_ 6m 



/ -cir2(c)<— ^ + — r(l). 
Jen c l-^f(e) 
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Therefore we can obtain an inequality on r(c) using (3.2 1 by 
-dr{c) = 



< 



< 



r 

J Co 


-d(r(c) 
c 




r 

J Co 


-dr2{c) 
c 


e 

+ - 
V 


14 


6m 


4 

— r 



€ 

v' 



The inequahty holds for arbitrarily large N and arbitrarily small e. We can 
conclude that 



f -dr{c) = [ -dr{c) < 1. 
Jo c Jn/2 c 



□ 



Remark. It is shown in the theorem that the achievable region of MRS codes co- 
incides with that of priority encoding transmission or multilevel diversity coding. 
When there are more than one transmitters, the MRS codes no longer admit the 
same region as priority encoding transmission or multilevel diversity coding in gen- 
eral. 



4. The Multiple Transmitter Setting 

In this section, we will discuss the case where there are d transmitters which 
cooperate to send the same sequence of data {Mi}, but they may or may not be 
transmitting the same sequence of encoding symbols. There are multiple receivers 
that wish to decode {Mi} sequentially at different rates. A receiver has an erasure 
channel connected to each of the d transmitters, and the channels may have different 
capacities. The central question of this section is that, given a set of receivers with 
different capacities and different rate requirements, is it possible to design a code 
which can satisfy the need of all receivers? 

We call the transmitters as Transmitter k, where k — l,...,d. We denote the 
symbol sent by Transmitter k at time n by Xk^n G {0, 1}. For a receiver with 
a channel from Transmitter k with capacity c, denote the symbol received from 
Transmitter k by Yk^c,n G {0,1, e}, and the indicator of erasure Ek.c,n € {Oil} 
iEk,c,n = 1 indicates an erasure). The definition of a multi-transmitter multi-rate 
sequential code is similar to that in the single transmitter case, and will be omitted. 

For a vector of channel capacities c G [0,1]^^, where Ck is the capacity of the 
channel to Transmitter k, we write 

Yc^n (^l,ci ,n 1 ■ ■ • 7 -^.Cd-n) 5 

-^c,n (-^l,ci ,n5 ■•■1 -^(i,Crf,Ti) • 

We use the notation Y^^^ — {Yc.a,YcM+i, ■•■,ic,b)- From now on, we refer to the 
receiver with channel capacities c as Receiver c. We write the sum of capacities in 
c by E(c) = c^■ 

Definition 8 (admissible pair). A rate-capacity pair (r, c) is called e-admissible 
by a code if Receiver c can decode the first N{r — e) bits M^^^ '^^ with bit error 
probability less than e when the first N symbols Y^^ are received, for sufficiently 
large N . More precisely, there exist Nq such that 

P {a/™ ^ M„ (Q, r,^i) } < e for any N > N^^m < N{r - e). 
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We use a function to characterize the rate of a code. We call r : [0,1]'^ [0, c») 
a rate- capacity function if it is monotonically increasing and right continuous along 
each of the d dimensions, and there exist an 77 > such that r(c) = for S(c) < 77. 

Definition 9 (rate of MRS code). A rate-capacity function r(c) is called e-admissible 
by a code if all of the pairs (c,r(c)) are e-admissible by the code. 

Definition 10 (achievable rate-capacity functions). A rate-capacity function r(c) 
is achievable if for any e > 0, there exist a code where r(c) is e— admissible by that 
code. 

The superposition MRS code for multiple transmitters is similar to that for single 
transmitter. 

Definition 11 (superposition MRS code). A superposition MRS code is char- 
acterized by the block size K and the parameter g : [0,oo) — > [0,c»)'^ which is 
bounded, monotonically decreasing and left continuous along each dimension with 
/o gk{ct)da = 1 for fc = 1, ...,d (write gk{c() for the fc-th entry of gica)). Transmit- 
ter k generates encoding symbols using the single transmitter superposition MRS 
code with block size K and parameter gfe(a). 

We say that a rate-capacity function is achievable by superposition MRS code if 
it is e— admissible by a superposition MRS code for arbitrarily small e. We give the 
necessary and sufficient condition on the achievability by superposition MRS code. 

Theorem 12. The rate-capacity function r(c) is achievable by superposition MRS 
code if and only if there exists a function g : [0,oo) — > [0,oo)'^ which is bounded, 
monotonically decreasing and left continuous along each of the d dimensions satis- 
fying 

gk{a)da = 1 for fc = 1, d, and 

c ■ g (r(c)) > 1 for any c G [0, 1]'' with r{c) > 0. 

Proof. The "if" part is similar to the proof of achievability in Theorem [7) Fix any 
e > 0. Let 77 > such that r(c) = for S(c) < r/. Define g' : [0, 00) [0, 00)'* by 

g'k{a) = {l + ^,)-gk{a + e/2), 

where ^fc > such that g'i.{a)da — 1. Define a new rate-capacity function 
r2(c) = max(r(c) — e/2, 0). We have, for any c with ^2(0) > 0, 

c • g' (r2(c)) = c • (1 + Off' (r-2(c)) > 1 + 

Consider the multiple transmitter superposition MRS code with parameter g{a). 
Applying Theorem [6] on each dimension, the rate-capacity function r2(c) is e/2- 
admissible by the code for sufficiently large block size, which implies that r(c) is 
e-admissible by the code. 

For the "only if" part, let e > and consider a superposition MRS code with 
block size K and parameter g{a) in which r(c) is e-admissible. Consider the receiver 
with capacities c which decode at rate r(c) — e. Using similar arguments as in [6j 
fixing any block with sufficiently large index, the expected number of times when 
a received symbol from Transmitter k is encoded from the block can be given by 
CkK ■ gk (f (c) — e). As at least K received symbols is required to decode the block, 
we have 

c • g (r(c) - e) = XI ^^^fe {r{c) - e) > 1. 

k 

As g{a) is left continuous, the proof can be completed by taking e — > 0. □ 
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5. General Non-optimality of Superposition Codes 

We have shown in Section [3] that superposition codes are optimal for single 
transmitter. However, in the multiple transmitter setting, the superposition MRS 
codes are not optimal in general. We will provide a counter example. 

Example 13. Consider the two transmitter case. Given the block size K and 
< ?■ < 2/3, the code is constructed by the following process. For Transmitter k 
(k — 1,2), at time i, we generate Xk^i by taking a random linear combination of 
the bits in the blocks ^^/^^^^^ _2+fc' K-lz/i4K)]-i-k^ and Ml^^^^^j^^^^^_^, where 

Consider Transmitter 1 , which generates AK encoding bits from the three blocks 

^2■\i/(iK)^-v ^i[i/{4:K)]~2' ^ili/{4K)]' "^i*^ ^ ^'^^^^ bits. Hencc, a 

receiver which can only receive from Transmitter 1 with channel capacity 3/4 can 
decode the three blocks. When [i/(4i^)] — n, the blocks M2„_i, M4„_2, and M!^^ 
can be decoded, which covers all the blocks. The rate-capacity pair (^,(f,0)) is 
admissible. Similar for (^7(0, |)). 

Consider a receiver which receives from Transmitter 1 and 2, each with chan- 
nel capacity 1/2. It receives 2K bits encoding the three blocks -M2 |"i/(4K)]-i' 
^l\i/{4K)^-2' ^'^l[i/(iK)V ^^^'^ ^^^^ encoding the three blocks M^.|-i/(4/f)^ , 

def 

^^ili/{iK)]-3' ^'^i[i/{iK)]-i- When n = li/iAK)] = 1, the AK bits encoding the 
blocks M[ to M4 are sufficient to decode the blocks. When n > 2, assume the 
blocks for m < 4n — 4 are already decoded, then there are 2K bits encoding 
the two blocks M4„_2 and M4„ (M2„_i is already decoded) which are sufficient to 
decode the blocks, and 2K bits encoding the two blocks M4„_3 and M^^_-^ which 
are sufficient to decode the blocks. The rate-capacity pair (l, (i, i)) is admissible. 

However, the rate-capacity pairs (5,(|,0)) and (1,(5,5)) cannot be simulta- 
neously achieved by superposition MRS code. If it can be achieved by superpo- 
sition MRS code, by Theorem [T2] there is a monotonically decreasing function 
g : [0,00) i-> [0,oo)2 satisfying gi{a)da = g2{a)da = 1 and lgi{^) > 1, 
i(7i(l) + i<?2(l) > 1. As gi(l),g2(l) < 1, we have gi{l) = 5-2(1) = 1, and 
51(0;) — 52(a) — 1 when a < 1, which contradicts with ^gi{-^) > 1. 

In the following sections, we will study some special cases in which superposition 
codes are optimal. 

6. Some Useful Tools 

We will present some tools which are used to find the admissible region in certain 
special cases. 

For the sake of simplicity, we write 

JNciyS) - i/(rc^i|M|^"J,£;5,g), and 

Jn^{YS) - /(Mfl^jVr^c^il^l''"^'<i'Q)- 

We use the infinity sign "00" in J^^O^c'i) the encoding symbols Xi are en- 
coded from and Q, and Fci can be determined by Xi and i?c,i- As a result, 

i/(i;^i|Mr,£;c^,Q) = o, 

and therefore 

^^.(^c^i) = i?(rc^i|Mf"J,<i,Q) 

= /(M^„j+i;i;^jA/L^"J,i?5,Q). 
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(Note that the support of Mf° is uncountable. The above equations only serve as 
the intuition behind the definition.) 

The quantity Jf^aO^I^i) roughly corresponds to the amount of information in 
the first N encoding symbols dedicated to encode the interval of data -^[ivfj+i- 
Furthermore, we define 

JaiYcT) = ^- £ JSie^i{Y}p)-e-dx, 

jiiY.^T) = ^.£j^^[::\iY}p).e-^dx. 
By observing H{Y^^\E^i) < N'E{c), we know the limits are finite as 



rl j'.l::\iy}p)-e-dx 



T 

< 



T 

< S(c), 

and thus 

(6.1) jf(yc,r)<s(c). 

Similarly, by considering [^x j (i^ij ^) < {P — a) [e^^J , we can obtain 

(6.2) jf(rc,T)</3-a. 

If (c, r) is e-admissible, then whenever r > /3 > a > 0, My^^^^^'^^^ can be decoded 
using Y^-y for sufficiently large N . By Fano's inequality (note that the case where 
/3 — e < a is obvious), 

r-T 



> liminfi./ {p-a-e){l-H{e))\_e^\-e 



= {p-a-e){l-H{e)), 

and therefore 

(6.3) liminf jf(yc,r) > - a - e)(l - Hie)). 
Also it is clear that for 7 < a < /3, 

(6.4) J^(Fe,T) = j"(yc,T) + jf(yc,T) 
The following lemma can be readily observed. 

Lemma 14. For c = ci + C2, ci,C2 > 0, we have 

ji{Y,„T) < ji{Y,,T) < ji{Y,„T) + J^{Y,,,T). 

Proof. As the quantities depend only on the marginal distributions of {Ec.i, Mi, Q}, 
{Eci,i,Xi,Mi,Q} and {Ec2,i,Xi,Mi,Q}, but not the joint distribution between 
Ec,i and -Bci,i and so on. For the purpose of analysis, we assume the non-erasure 
positions of Receiver Ci do not overlap with those of Receiver C2, and erasure hap- 
pens in Receiver c if an only if erasure happens in both Receiver Ci and Receiver 
C2. Then Y^^ has the same information as {Y^ i,Y^ i). We can deduce from 
I{X; Z) < l(x, Y; Z) < I{X; Z) + H{Y) that 

tNI3(vN \ ^ jNl3(T^N\ ^ jNficyN \, too (^N \ 
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The result follows. □ 

As the message is transmitted in a sequential manner, the received symbols Y^^ 
should contain more information about the older messages (the Mj with smaller i) 

than newer messages, and therefore the average information (/3 — q)^^ ■ J^^{Yc,T) 
should increase when a decrease. This property is proved in the following lemma. 

Lemma 15. Let < 7 < a < /9. We have 

hminf - J^d^.T) - ^-jtiYcT)) > 0. 

Proof. Consider 

"^7Le==J V-^c.l ) — '^aLe'J c,l ) + -f[e^ ] \^ c,l ) 

> j'J::]iY}p)+j^::{Y}:P)-a. 



L 



After integrating the second term, we get, for any Xq, 



> / j^!:{Y}:/^)dx 

J Xo 

/■xo+log(^/a) rxo+los{0/a) 

= / J^AY^l^)dx~ JZ4y1:, ^)dx 

Jxo Jxo 

/.a;o-log(Q/7)+log(/3/a) [e'o \ rxo+logi/3/a) [e^o i 

= / ■^^x+iog(a/7) (^c,l )dx — / ■^S»(^c,l )dx 

J xo-\og{a/^) Jxo 

/■xo+log(/37/a^) ^ i-xo+\og{P/a) ^ 

= / JZ^ {Y^^ °^)dx- {y};, °i)dx 

J xo—\og(a/f) Jxo 
r° 00 Le""J rxo+los{i3/a) ^ ^^^^^ ^ 

= / "^ae^(^c,l )dx — / JZe^iYc,! )dx 

Jxo-log(a/7) J xo+\og{P^ / a^) 

pxo fXo 

= / JZ^{Y^r^)dx^ / J^AY^r^)dx 

rxo 
J X 



/xo-log(a/7) 

> /" ttAY^P)dx 

Jxo-log(a/7) 
rxo 

^ / •^aLe«j(^c,i )«^a;-alog- 

J xn — los.{aH) 



•^ae^O^cA ^)dx 



'xo-log(a/7) 

As a result, 

PXo 

J X, 



xo-log(/3/a) 



> / jZ^\iY^l^)+ j^!:{Y^V)-c.\og^- 

Jxo-log(/3/a) Ja;o-log(/3/a) " 

> / J%\^Y^l')+ jtU^Y^l')dx-aXo^^- 

Jxo-logifS/a) ixo-log(/3/7) 7 

(0^5) > f .f[:;j(i;'rJ)-«.ogf 

Jxo-log(/3/7) r 
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Fix any T > 0. For each term in (6.51, multiply it with e ^° and integrate it 
from to T, we get 

a:o-log(;3/a) 
T /.min(T,x+log(/3/a)) 



10 Jx 

rT /.x+log(/3/Q) 



< 



^^y,.l{Y^:^')dxodx 



And also 

rT 







> 



fX.\{Y^P)dxdx, 

2;o-log(/3/7) 
T /.min(T,a:+log(/3/7)) 



J X 

T-log(/3/7) .£>;+log(/3/7) 



= 1 



> 



1 



T-log(^3/7) 



e-jZ.\{Y^P)dx 



e-jZ.\(Yy-^)dx- 



1 - 



(/3-a)log 



Therefore, by (6.5 1, 



1 



> 



e-jZ\{Y^:;^)dx 



1 - 



e-^jff::i(^i,fj)d- 



- 1- 



(/3-a)lof 
(/3-a)loj 



7 
7 



e ^"alog-dxo 
7 

(l — e^^) a log — 



> 



(/3-a)(/3-7) 
/3 



a • loE 



Multiply /3/ (T • (/? — a)(/3 — 7)) to both sides, we can obtain 



> 



1 



1 



T 



1 



(/?-a)(/3-7) 



log 



7' 



Note that the second term vanishes when T ^ 00. The result follows. □ 

We now proceed to prove an inequality on achievable rate-capacity functions. 

Lemma 16. // a rate-capacity function r(c) is achievable, then for any c = Ci + 
C2 + ... + c„, where > and r(c) > r(cfe) for k — 1, n, we have 



S(cfc) - r(cfc) ^ ^ 



r(c) - r(cfc) 
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Proof. Without loss of generality, assume ^(ci) < ... < r(c„) < r(c). Let Cq = 0. 
Consider a code in which r(c) is e— admissible. Fix any k E {1, n}. By (6.1 ), for 

any T > 0, 

Jr(^c.,r)<s(cfc). 



By (je^, 

Invoking Lemma [T4j we obtain 

> j:[11){y,,+...+c,,t)-j:{c,) + jI^''''\y^,,t), 

It can be deduced using Lemma [T5| that 



^(c)-r(cfe)'^'(2)(^c,+...+c._,,T)j > 0, 



and therefore 

/ \ / \ "J r(c.,. 



lim inf 

T-»-oo \ r(c) — 



> 0, 



Summing through k ~ 1, ...^n, 



and thus 

7'"(cfc) 



(>^c.,T) 



Using (6.3 1, we have 



and 



liminf J^^'^^VcT) > (r(cfe) - e) (1 - H{e)) , 

T— >oo 



Inninf J:£)(yc, T) > (r(c) - r(c„) - e) (1 - H{e)) 
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Hence, 



E 



Let 



S(cfc) - (r(cfe)-e)(l~g(e)) 
fc^i r(c)-r(cfc) 

(r(c) ~ r(c„) - 6) (1 - 
r(c) - r(c„) 
0. We obtained the desired result as 
S(cfe) - r(cfc) 



> 0. 



E 

fc=i 



r(c) - r(cfc) 



> 1. 



□ 



We will discuss some cases in which superposition coding is provably optimal in 
the next section. 

7. On-Off Multicast Networks 

In this section, we consider networks in which there are d transmitters and 
2'^ — 1 receivers, each having a different set of transmitters to which it is connected. 
Transmitter k broadcasts the same information to the receivers it is connected to 
at rate Wk- Each receiver has to decode the information at a different rate. There is 
no erasure in the network. We would like to formulate the criteria on the decoding 
rates of the receivers in which sequential data transmission is possible. 

We first convert the problem into the multi-transmitter MRS setting. Without 
loss of generality, we assume Wk < 1 for k ~ l,...,d. We may replace a connec- 
tion with rate w by an erasure channel with capacity w. Note that the decoding 
requirement does not depend on the joint distribution of erasure events of differ- 
ent receivers. Therefore, the problem can be translated to d-transmitter MRS. We 
confine our study to the capacity vectors c e {0,wi} x ... x {0,^^}. 



Source 



Transmitters 



Receivers 




Figure 7.1. A three-transmitter on-off multicast network 



7.1. Non-optimality of superposition coding in general. We can construct 
a network in which superposition coding is not optimal, using a similar idea as in 
Example [13] 

Example 17. The network contains 3 transmitters and 3 receivers, where Trans- 
mitter 1 and 2 broadcast at rate 1, and Transmitter 3 broadcast at rate 2. Receiver 
1 is connected to Transmitter 1 and 2. Receiver 2 is connected to Transmitter 1 
and 3. Receiver 3 is connected to Transmitter 2 and 3. Non-superposition code 
can achieve the decoding rate 3/2 for Receiver 1, and 3 for Receiver 2 and 3, which 
are not achievable using superposition codes. Please refer to Subsection |7.3| for the 
proof of achievability and further discussions. 
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Source 
Transmitters 

Receivers 




3/2 3 3 



Figure 7.2. The network in which superposition codes are not optimal 



Although superposition coding is not optimal for networks with 3 transmitters 
in general, it is optimal for networks with 2 transmitters, which will be shown in 
the following example. 

7.2. One-or-all on-off multicast network. 

Example 18 (One-or-all on-ofF multicast network). There are d transmitters and 
d + I receivers (numbered 0,...,c?), where Transmitter k broadcasts the same infor- 
mation to Receiver and k at rate of Wk bit/s for k ^ I, d. Receiver k has to 
decode the data at rate bit/s for k — 0, ...,d. We would like to characterize the 



achievable region of {r^} in terms of {wk}- Figure 7.3 shows the network when 
d = 2. 



Source 
Transmitters 
Receivers 




Figure 7.3. The two-transmitter one-or-all on-off multicast network 

The following theorem shows that superposition coding is optimal. 

Theorem 19. Superposition coding is optimal in the one-or-all on-off multicast 
network, which has an achievable region 

ro >0, <rk < Wk for fc = 1, d, 

d 

either ro < nia.x(wk) or } — > 1. 

ti '^^"-^ 

Proof. The case where tq < max(w7fe) is trivial. We assume > for all k. 
Without loss of generality, assume ri < ... < r^, then we have ri < ... < r^ < Wd < 
ro- 

Among the inequalities in the proposed achievable region, rj, < Wk is obvious, 
and ^ > 1 is due to Lemma 16 The converse follows. 

^k=L m — ri, — 
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We now show superposition MRS code can achieve the region. Assume "ra-rk — 

1. Consider the parameter 



Note that 



when a < rk 
qJa) = { ^ when ri. < a < rn 

when a > tq. 



Wk-Tk ^1 



Wk{ro - rk) Wk 

due to To > Wk- Therefore g{a) is monotonically decreasing along each dimension. 
It can be easily checked that gk{a)da = 1. It is left to check c • g{r{c)) > 1. 

Wk ■ gk{rk) = Wk ■ — = I, 

Wk 

d 

■ gk{ro) 



k=l 



EWk - rk 
Wk- 



Wk{ro - rk) 
d 

Wk - rk ^ ^ 
f-^ro - rk ~ 

k—l 



By Theorem [T2j the region is achievable by superposition MRS code. □ 

Remark. The achievable region in the MRS setting is different ft'om that in multi- 
level diversity coding, which admits the larger region 

d d 

ro>0,0 <rk<Wk for k = 1, d, + rj. - max (r^) < Wk- 



fc=i fc=i 



7.3. Non-superposition codes. In Example 17 we presented a network in which 
superposition codes are non-optimal. Even though non-superposition codes are 
used, the tools described in Section [6] may still be used. We are going to prove the 
achievable region of a generalized version of Example |17[ 

Example 20. There are 3 transmitters and 3 receivers numbered 1,2,3. Receiver 
1 receives from Transmitter 1 and 2, and wish to decode at ri. Receiver 2 receives 
from Transmitter 1 and 3, and wish to decode at r2 > ri. Receiver 3 receives from 
Transmitter 2 and 3, and wish to decode at r^ — r2- 



Source 
Transmitters 



Receivers 




Figure 7.4. The network specified in Example [20] 
The achievable region is given by the following theorem. 
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Theorem 21. The achievable region in Example 20 can be given by 

Wi+ W2 > ri 

W2 + > r2 

f2 ^ fl 

Wl + K72 + W3 • > r2- 

r2 

Proof of converse. The first three inequahties clearly hold. For the last inequality, 
consider a code in which the rate requirements are e— admissible. Receiver 1 can 
decode at rate ri. By ( [6.3^ , 

J2{Y(^,,n,..,o))>{ri-e){\^H{e)). 

Invoking ( |6.1| and Lemma [l4| we can obtain 

5^0^(^(^)1,0,0)) + ■"'2 > iZo^(i^(u.i,0,0)) + -'^O (^(0,ii;2,0)) 

(7.1) > (ri-6)(l-iJ(e)). 



Receiver 2 can decode at rate r2. By (6.3 1, 

Z;?(i"(»i,o,^3))>(^2-ri-e)(l-i?(e)). 
Invoking Lemma [14] we can obtain 

•^n 0,0)) + i^ri (^(0,0,11)3)) > iZri(^(«;i,0,'i«3)) 

> (r2 - ri - e) (1 - 



By Lemma 15 and m.l 



r2 



and therefore 
(7.2) 



(1^(^1.0,0)) + ^^^3^ • ^^3 > (r2 - n - e) (1 - i/(e)) 



r2 



By (6.1) and (^6.4 



(7.3) 



wi > ^ (y(t«i,o,o)) 

> iZo^(^(ii;i,o,o)) + >^ri (^(u;l,o,o))■ 



Adding ([TJJ), (|7^ and ( |7.3| , 

+ W2 + '"^ ''^^ • W3 > (r-2 - 2e) (1 - i/(e)) 

?'2 

The proof can be completed by taking e — J' 0. 



□ 



Proof of achievability. We will describe a coding scheme which can achieve the 
proposed region. When ri = r2, we may use all the transmitters to transmit at 
rate ri. When W3 > r2, then we can use Transmitter 3 alone to transmit the 
message at rate r2, and Transmitter 1 together with Transmitter 2 to transmit at 
rate ri. Therefore we assume ri < r2 and W3 < r2. 

The code is specified by the block size K, the super-block size L > K, and 
the parameter 7. Divide the message {Mi}^^^ ^ into blocks of K bits Bi = 
■^(i^i)K+i- Further divide each block Bi into two sub-blocks Bi i and B2.i, where 
Bi i contains the first "/K bits of the block, where 7 is taken to be 'W3/r2 in this 
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case, and B2 i contains the rest of the bits. Group the sub-blocks into super-blocks 

by 

Sa,b,i = {Bb,{i-l)-ra-L/K+l, BbA-ra-L/K) 

for a,b £ {1,2}. Assume ri • L/K and r2 -L/K arc integers. Each super-block Sa,i,i 
contains • bits, and each super-block Sa,2,i contains Tq • (1 — j)L bits. 



Transmitters 1,2 



Transmitter 3 

^2,1.6 



2/3 
1/3 



— 1 






^1,1,. 


\ 






s s 

:j.MiS 1,1,10 


s 

1.1,11 


s 

1,1, 1i 




.2,1 


S2 


2,2 




2,3 


^2,2,4 


J ^2,2,5 


S2 


2,6 















Figure 7.5. The super-blocks when 7 = 2/3 and r2 = 2ri 



At time i, Transmitter 1 encodes the super-blocks S'i,i,[i/L] and 82^2, [i/L] using 
random linear projection (concatenate the bits in the super-blocks and transmit a 
random projection of the resultant vector). Transmitter 2 uses the same encoding 
scheme as Transmitter 1. Transmitter 3 encodes the super-block 52,1, [i/i,] using 
random linear projection. 

To see why Receiver 2 can decode at rate r2, assume that at time jL, the super- 
blocks S'2,i,fc and 5'2,2,fc are already decoded for k= 1, During the time interval 
jL + 1, (j + 1)L, Receiver 2 will receive w^L bits from Transmitter 3, and wiL 
bits from Transmitter 1. For a bit encoded by Transmitter 1 at time i, as the 
super-block Si^i^^i/^-^ is already decoded (for j large enough), it can be treated 
as a random linear projection of 5*2,2. ["i/L] • Therefore all together we have w^L 
projections of ^2, 1,^+1, and wiL projections of S2,2,j+i- By definition of 7, 

wsL = r2'yL. 
From the assumption wi + > r2, 

wiL > (r2 — u>3)L 
= r2(l-7)i. 

Therefore both 82,1. j+i and 52,2,j+i can be decoded at time {j + l)K/r2- By 
induction. Receiver 2 can decode at rate r2- (Note that we may assume the first 
few blocks are decoded, as we may allocate any extra amount of time to transmit 
them without affecting the asymptotic behavior of the code). Similar result holds 
for Receiver 3. 

For Receiver 1, consider the time interval {j-\-l)L. In this time interval. 

Receiver 2 receives W\L bits from Transmitter 1, and W2L bits from Transmitter 2. 
All together there are (wi + W2)L projections of and 52,2,j-i-i. The number 

of bits in 5i,i,j+i and 52,2,j-i-i is given by 



ri • 7L -I- r2 • (1 - 7)L = 



riw;3 



r2 

= I r2 - W3 



r2 - w;3 1 • 

r2 - ri 



r2 



■L 
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which is smaller than {wi + W2)L by the assumption wi + W2 + W3 ■ ^^j.J'^ > ^2. 
Therefore Receiver 1 is able to decode at rate ri. □ 



Remark. Superposition codes can achieve a smaller region given by 

> n 

> r2 

> r2 

> 2r2-ri. 

8. Conclusion and Discussions 

In this report, we have investigated the achievable regions and coding schemes for 
multi-rate data transmission. We have shown that superposition codes are optimal 
for the single transmitter setting. However, in the multiple transmitter setting, 
there are some non-superposition codes which outperform superposition codes. 

Our results can be applied in various scenarios which requires the messages to 
be decoded sequentially, for example, the broadcast streaming of video. The multi- 
rate sequential data transmission setting can also be applied on messages divided 
into several levels of importance. For example, in the transmission of an interlaced 
image file, the data corresponding to the low-resolution part is transmitted before 
the data corresponding to the high-resolution part. A sequential code can ensure 
that, even when the receiver has variable channel condition, the low-resolution 
part is decoded first, and therefore the receiver can display the image with lower 
resolution before all data are received. If the connection might be stopped at any 
time, using a sequential code can ensure the received message forms a continuous 
segment from the beginning instead of fragmented data as in Fountain codes. In 
the example of image transmission, if the connection is lost in the middle of the 
transmission, the receiver can still decode a low-resolution version of the image. 

In Example |20[ we have studied a particular 3-transmitter network. Further in- 
vestigation on the general 3-transmitter network, and the next step, A^-transmitter 
networks, may be carried out in the future. Ultimately, we may consider general 
networks of interconnected nodes instead of only two layers of nodes (transmitters 
and receivers) . Another direction is to find a method to construct non-superposition 
codes according to the network connections and decoding rate requirements. Ex- 
ample |20] presents the construction of a non-superposition code using a sub-block 
structure. It is left for future studies to find out whether this construction method 
give the optimal code in more general settings. 
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